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Abstract 

We propose a novel method of efficient upsampling of a 
single natural image. Current methods for image upsam¬ 
pling tend to produce high-resolution images with either 
blurry salient edges, or loss of fine textural detail, or spuri¬ 
ous noise artifacts. 

In our method, we mitigate these effects by modeling the 
input image as a sum of edge and detail layers, operating 
upon these layers separately, and merging the upscaled re¬ 
sults in an automatic fashion. We formulate the upsampled 
output image as the solution to a non-convex energy min¬ 
imization problem, and propose an algorithm to obtain a 
tractable approximate solution. Our algorithm comprises 
two main stages. 1) For the edge layer, we use a nonpara- 
metric approach by constructing a dictionary of patches 
from a given image, and synthesize edge regions in a higher- 
resolution version of the image. 2) For the detail layer, we 
use a global parametric texture enhancement approach to 
synthesize detail regions across the image. 

We demonstrate that our method is able to accurately 
reproduce sharp edges as well as synthesize photorealistic 
textures, while avoiding common artifacts such as ringing 
and haloing. In addition, our method involves no training 
phase or estimation of model parameters, and is easily par- 
allelizable. We demonstrate the utility of our method on a 
number of challenging standard test photos. 

1. Introduction 

The last few years in imaging technology has witnessed 
the advent of ubiquitous high-definition (HD) image dis¬ 
play systems, as well as an exponential growth in low- 
resolution image acquisition sytems, such as cellphone 
cameras. Given this scenario, the classical problem of 
image upsampling assumes a renewed significance. Im¬ 
age upsampling (also variously termed as upscaling and 
super-resolution ) is a highly ill-posed linear inverse prob¬ 
lem, since the number of unknowns (high-resolution im- 
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Figure 1. Various stages in our upsampling framework. Bicubic 
interpolation creates an initial estimate h (top-left), which is fur¬ 
ther refined using a localized nonparametric patch synthesis to ob¬ 
tain a second estimate h (top-right). While this estimate repro¬ 
duces smooth regions and sharp edges, it suffers from a noticeable 
lack of detail. Therefore, it is decomposed into an edge and detail 
components, and an ^-shaped curve is applied to achieve a global 
detail enhancement. To prevent exaggeration of noise and other 
artifacts, we use a blending o-mask (bottom-left), calculated dur¬ 
ing the synthesis step, to selectively enhance only the detail layer 
and thus obtain the final image I (bottom-right). 


age pixel values) exceeds the number of observations (low- 
resolution pixel values) by an order of magnitude, even at 
moderate upscaling factors. The challenge is exacerbated 
by the unique difficulties encountered in modeling real- 
world photos, as well as the inevitable presence of added 
nuisances such as camera blur and noise in captured images. 

Broadly, image upsampling methods can be divided into 
parametric and non-parametric approaches. Methods be¬ 
longing to the first category assume parametric models for 
the unknown high-resolution image. The prominent method 
in this class is the familiar bicubic interpolation methods 
(that assumes a bandlimited structure on images), and is in- 
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deed the standard algorithm in commercial photo editing 
packages such as Adobe Photoshop. Over the last 10 years, 
more sophisticated parametric models have been developed. 
Total variation minimization methods assume that images 
have bounded TV-norms [1]); probabilistic models assume 
a prior on gradient profiles of images [10, 11, 4]), and spec¬ 
ify (or estimate from data) the hyperparameters of these pri¬ 
ors. Sparse models assume that image patches are sparse in 
a basis or learnt dictionary [15, 14]). 

Methods belonging to the second category do not as¬ 
sume an explicit model for images or image patches; instead 
they exploit natural image invariances such as scale- or 
translation-invariance; in such cases, the prior for the high- 
resolution unknown image patches are raw patches from a 
database of external images [6], or even patches from the 
scale-space input image [2, 7]. These methods ‘learn’ cor¬ 
respondences between high-resolution and low resolution 
patches and store this in a searchable database; during re¬ 
construction, every occurrence of a low-resolution patch is 
replaced by its corresponding high-resolution patch. 

Both types of approaches typically possess one or more 
of the following drawbacks: 

Loss of fine details Neither category of methods seems to 
reproduce realistic textural details for even moderate up- 
scaling factors. 

Visual artifacts The resulting images suffer from various 
artifacts such as blurry edges, exaggerated noise, halos, 
ringing artifacts, and staircases (“jaggies”). 

Computational demands All state-of-the-art methods in¬ 
volve considerable computational costs. Upsampling a 
small (128 x 128) image by a factor of 4 might typically 
take several minutes. 

In this work, we develop a new hybrid algorithm for im¬ 
age upsampling which greatly alleviates each of these con¬ 
cerns. The key idea behind our approach is that sharp edges 
and fine details in natural images exhibit inherently differ¬ 
ent structure , and hence should be separately upscaled. We 
model the output image as the linear sum of edge and detail 
layers and pose the upsamping problem as a highly under¬ 
determined linear inverse system of equations. We regular¬ 
ize this problem by imposing the image-specific prior that 
all patches of the target image can be well-approximated by 
the patches extracted from the low-resolution observed im¬ 
age [2]. Thus, we obtain an optimization formulation of the 
upsampling framework. 

We show that this optimization is highly non-convex, and 
indeed intractable. To resolve this, we devise the follow¬ 
ing approximate algorithm. First, we use bicubic interpo¬ 
lation to obtain an initial high-resolution estimate. Next, 
we greedily replace each patch in the high-resolution im¬ 
age by a weighted sum of “similar” patches from the low- 
resolution patch manifold; we argue that this step both re¬ 
constructs the edge layer, as well as provides an a-map in¬ 


dicating what pixels constitute the detail layer. Finally, to 
restore the fine textural details in the image, we decompose 
this intermediate image into its edge and detail layers using 
an edge-aware filtering technique [ ], and selectively en¬ 
hance the detail layer using a parametric pixel-wise remap¬ 
ping function. See Fig. 1 for an illustration of our algorithm. 

We demonstrate the utility of our method on a number 
of challenging standard test photos. Our method consis¬ 
tently improves upon state-of-the-art results at much lower 
computational costs. The algorithm involves no training or 
learning of model parameters, and does not need an external 
database as input. Additionally, our method is fast and par- 
allelizable, and lends itself to easy potential implementation 
for very high-definition images and videos. 

2. Layer-Based Upsampling 

2.1. Setup 

Consider a given low-resolution nxn image . Our goal 
is to obtain a photorealistic high-resolution natural image 
I whose downsampled version is equal to (or close to) 

The image I can be conceptually decomposed into edge and 
detail layers: 

I = E + D , (1) 

where the edge layer E is piecewise smooth, and the detail 
layer D consists of high-frequency variations of small mag¬ 
nitude. Figure 2 provides a simplified illustration of this 
model in ID. If C denotes the linear downsampling opera¬ 
tor, then the observed low-resolution image / is given by: 

I*=C{I) = C{E + D). (2) 

Equation 2 represents a highly underdetermined system of 
linear equations comprising n 2 constraints and 2 a 2 n 2 un¬ 
knowns, where a > 1 is the desired upsampling factor. 
Consequently, this formulation is ill-posed. 

2.2. Optimization via the Patch Transform 

How do we regularize this problem? Given an image 
/, denote the patch transform P{I ) as the set of (overlap¬ 
ping) r x r patches extracted from the image. Obviously, 
the patch transform P is an invertible mapping; ignoring 
image boundary effects, every pixel in the image occurs in 
r 2 elements in P. For completeness, we denote the (inverse) 
linear synthesis as I = S(P). We trivially observe that any 
pixel intensity at a location x in the image / can be repre¬ 
sented as the mean of the intensity values of the r 2 patches 
that contain the pixel x, i.e., /(x) = mean(p(x)), p G 
P(7). We also trivially observe that the variance of the r 2 
different explanations of the pixel intensity (in short, the 
pixel variance) at x is zero. 

Due to the one-to-one nature of the patch transform, any 
prior on the patch transform of an image is equivalent to a 


modified prior on the image itself. Then, an intuitive regu¬ 
larization for ( 2) would be to require that the patch trans¬ 
form P(I) should be a subset of the set of natural image 
patches A4, since it is well known that the set M. occupies 
an extremely tiny fraction of the space of all possible r x r 
patches. However, this constraint itself is unwieldy since 
there exists no known concise parametric or non-parametric 
model for A4 ; indeed there is no consensus for what char¬ 
acterizes a natural image. 

Therefore, we adopt a more tractable relaxation of (2): 
the patch transform of the target image P(I) must be con¬ 
tained in the patch transform of the low-resolution image 
P(J 0 ). This hypothesis is justified by several recent im¬ 
portant findings in the literature [16, ] that natural image 
patches tend to be self-similar in highly localized regions in 
scale space. Therefore, our desired upsampled image can 
be posed as the solution to the following optimization: 

I = argmin ||/ 0 ~£(/)|[, (3) 

s.t. / = S(P), and P(I) C P(/ 0 ). 

2.3. Challenges 

The above optimization poses two primary caveats. The 
first caveat is computation complexity. Equation (4) is a 
highly non-convex, combinatorial subset selection problem, 
and therefore is very difficult to optimize exactly. However, 
there potentially exist fast approximate schemes to obtain a 
suboptimal solution, one of which we will propose in Sec¬ 
tion 3. 

The second caveat arises due to a subtle modeling er¬ 
ror — importantly in practice, that the patch transform of 
the low-resolution image Io only contains the patch trans¬ 
form of the edge layer P(E ), and cannot explain patches 
from the detail layer D. Indeed, as extensive experiments 
with natural images in [1( ] have shown, the scale invari¬ 
ance property for natural images holds for smooth regions 
and sharp edges, but not for fine textural details. When the 
detail layer D = 0, we can conceptually obtain a precise, 
consistent solution to (4). However, when D ^ 0, differ¬ 
ent patches in P(l ) containing the location x would offer 
inconsistent explanations of the pixel value at x. Conse¬ 
quently, the variance of the different pixel explanations at x 
is high, and the detail pixels gets “averaged out”; this phe¬ 
nomenon is corroborated in our numerical experiments. 

We propose a greedy nonparametric scheme to solve the 
above optimization problem. In the course of this scheme, 
we also detect which pixels possess high pixel variance; 
these correspond to the detail pixels. We then employ a 
simple detail enhancement heuristic to reconstruct the de¬ 
tail pixels, simultaneously making sure that the final result 
is consistent with the input low-resolution image. The full 
algorithm is as discussed below. 
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Figure 2. Proposed image model illustrated in ID. The image I is 
modeled as the sum of an edge layer E and a detail layer D. 

3. Proposed Upsampling Algorithm 
3.1. Description 

Our algorithm consists of two duelling subroutines 
that are applied in an alternating fashion. We first esti¬ 
mate the patch transform of the desired image using the 
non-parametric approach: given a blurry high-resolution 
patch, we extract a few similar raw patches from the input 
low-resolution image and replace the blurry patch with a 
weighted combination of the extracted raw patches. Next, 
we perform an additional edge-detail decomposition of the 
estimated image, and carry out a pointwise remapping to 
enhance only the detail pixels. A careful recombination of 
the outputs of this decomposition results in a photorealistic 
final image that consists of sharp edges, fine scale details 
and no visible artifacts in smooth regions. For the rest of 
the paper, we assume that we are working with intensity 
(luminance) images with dynamic range [0,1]. 

Estimation of the edge layer Given a low-resolution 
image Jo, we perform an initial upsampling by a small 
factor using a fast parametric interpolation (we use bicubic 
interpolation in all our experiments.) If U denotes a linear 
interpolation operator, then the image 

I = U(I 0 ) 

represents our initial intermediate estimate of the upsam¬ 
pled image. However, this parametric interpolation results 
in reduced sharpness of edges, resulting in a degradation in 
quality. We compensate for this as follows. Given a blurry 
high-resolution r x r patch p(x) C I (where x denotes the 
spatial location), we perform a localized search for raw rxr 
patches in the given low-resolution image 7 q (r = 5 in all 
our experiments). Formally, we perform a nearest-neighbor 
search: 

q = arg min d 1 (p',p) + A d 2 (x', x), (4) 

p'(x')e/ 0 

where d\ is the distance between the patch intensity values 
and ^2 represents the distance between the spatial locations 
of the patches (normalized with respect to the size of the 
image). We choose this distance function since it penalizes 
both In the standard manner, we remove the DC component 










Io i I- 

Figure 3. Various stages of the proposed upsampling process illus¬ 
trated in ID. The input image /o, the downsampled version of the 
image I in Fig. 2, consists of both sharp edges and detail patches. 
An initial bicubic interpolation provides an estimate I which con¬ 
sists of blurry (unsharp) edges and reduced details. Replacing ev¬ 
ery patch in the patch transform of I by its nearest neighbor in 
the patch transform of Io produces the nonparametric estimate /, 
which has sharp edges and reduced details. 

of each patch before performing the nearest neighbor opti¬ 
mization. In our experiments, we set a value of A = 10. 

Once a few ‘close’ example patches qj have been found, 
we replace the patch p with a weighted combination of the 
extracted raw patches. Overlaps between resulting patches 
q are further averaged together to reduce blocking artifacts: 

I{ X ') = Yl ^' W 1o' G <?’ ( 5 ) 

j:x'eqj 

where the Gaussian weights G a favor greater weights to the 
center of the 5x5 patches q (a = 1.25 in all our exper¬ 
iments), and the weights w q = di(q : p)~ 1 favor greater 
weights to better matches to the image. Note that Eq. 5 ef¬ 
fectively calculates a weighted version of the synthesis step 
<S, the inverse of the patch transform of I . Thus, if A f de¬ 
notes the overall non-parametric synthesis procedure, then 
we have that 

i = N{i). 

Thus, we obtain a second intermediate image / with sharp, 
realistic-looking edges. 

In this manner, we have simulated the optimization in 
Eq. 4. However, as argued, the obtained estimate I suffers 
from a loss of detail in textured regions. The reason is that 
the small, fine-scale variations present in textured regions 
in the high-resolution target image do not manifest them¬ 
selves in their lower-resolution counterparts. Therefore, 
they cannot be synthesized by a simple non-parametric 
search and are averaged out in the synthesis step S (Eq. 5). 

Estimation of the detail layer To solve this problem, 
we perform a global detail enhancement on /. We decom¬ 
pose / into a piecewise-smooth component and a detail 
component, akin to the model shown in Fig. 2, using the 
weighted least-squares (WLS) approach [3] 1 : / = E + D. 

! We chose this method since it generates high quality decompositions 
in linear time. Other decompositions, such as the bilateral filter [12] may 
equally be used. 



Figure 4. Detail-enhancing pointwise pixel remapping functions 
/(A) = A^ for various values of (3. Smaller values of (3 indicate 
a higher amount of detail enhancement. 

Next, we apply a detail-enhancing pixelwise remapping 
function: 

/(A) = A*, 

to obtain a new detail layer image f(D). Informally, we 
call this function an “S-curve”; see Fig. 4 for some example 
remapping functions /(•). We will discuss how to choose 
the scalar parameter /3 below. 

This step restores fine-scale details, but also aggravates 
noise and artifacts in smooth and edge regions. Therefore, 
while synthesizing the final image, we only consider the 
pixels that have been “excessively averaged” in the previ¬ 
ous step. This is easily achieved by computing the variance 
of different explanations for a pixel in the patch transform 
synthesis (Eq. 5), and constructing a binary a-mask that in¬ 
dicates the pixels with high pixel variance (Fig. 1). We 
selectively blend in these pixels from the enhanced detail 
layer using the equation: 

I=(l-a)I + af(D) = £(I), 

where £ denotes the overall detail-enhancement operator. 
This final step results in a high-resolution image I with 
sharp edges and a requisite amount of details. Thus, the 
overall upscaling procedure can be summarized by the com¬ 
posite operator relation: 

I = £(Af(U(I 0 ))). ( 6 ) 

Figure 3 illustrates (on a simplified ID signal) the results 
of some intermediate steps in our proposed algorithm. The 
bicubic upsampling step U yields poor results on both 
edges and textures; the non-parametric synthesis step J\f 
corrects for the edges, but suffers from a lack of details. 
Figure 1 illustrates the results on a standard test image, and 
serves to highlight the importanc of the detail enhancement 
step £ within our framework. The step £, together with 
an intuitively chosen blending function a , ensures that 
relevant and photorealistic details are added. 

Consistency An important consideration is to ensure 







P = 1.0 p = 0.9 p — 0.8 p = 0.7 p = 0.6 

Figure 5. Example (cropped) 3x upsampling results of a 240 x 160 image using layer-based upsampling. A value of P — 0.8 gives visually 
pleasant results for a large set of images. 


that each of the upsampling operators results in an image 
that is consistent with the low-resolution observed im¬ 
age. We achieve this by running a few iterations of the 
linear back-projection step [9] (commonly used in many 
upsampling schemes) for each intermediate estimate / and 
/. This step is equivalent to finding the closest image in 
the subspace of feasible high resolution solutions to the 
intermediate image, thus ensuring a final estimate / that is 
consistent with the low resolution input image Iq. 

3.2. Discussion 

Implementation details In practice, the above upsam¬ 
pling process gives desirable results for upscaling factors 
< 1.25. This is due to the fact that the scale-invariance 
hypothesis is truly valid only for small scale factors. To 
achieve any desired magnification factor, our procedure is 
repeated several times (e.g., to achieve a upscaling factor of 
4, the above three-step approach is performed by applying 
a scale factor of 7 = 2 1 / 3 a total of 6 successive times.) 

In all our experiments, we perform the above upsam¬ 
pling by first transforming the image into YCbCr space, 
and applying the upsampling to the luminance channel. 
This is due to the well-known fact that visual perception of 
edges and details responds primarily to the luminance of 
the image. 

Parameters In sharp contrast with parametric upsam¬ 
pling methods such as imposed edge statistics [4] and 
gradient profile priors [ 11 ], our method does not involve 
any training phase involving estimation of different pa¬ 
rameters. In our experiments, we have observed that our 
method is robust to the choice of the number of interme¬ 
diate upsampling steps 7; further, it is also robust to the 
choice of the number of nearest neighbors obtained in the 
nonparametric search k, as well as the choice of the weight¬ 
ing parameter A that controls the tradeoff between accuracy 
of nearest neighbors versus the locality of the search. The 
only changing parameter in our different experiments is the 
exponent P used in the detail enhancement step; this varies 
depending on the amount of detail present in the original 
image, as well as the sharpness of desired detail in the 


upsampled image. A smaller value of P corresponds to a 
‘more-acute’ S-curve, and hence exaggerates finer details. 
Figure 5 illustrates the effect of varying the parameter P 
across a range of values (/? = 1.0 implies that no detail 
layer exaggeration is performed). For small values of 
P, we notice that the details get artificially exaggerated. 
Unless otherwise stated, we use the value P = 0.8 in our 
experiments; in our experience, this choice yields visually 
pleasant results. 

Computational advantages Our proposed method 
does not involve any external database; instead, it only re¬ 
lies on self similarities present in the input image. Further, 
it requires no special learning, training, or tuning of model 
parameters. Moreover, our method obviates the need for an 
exhaustive search of the image’s scale space. To synthesize 
each upsampling estimate, we only operate upon pixels 
and patches belonging to the current intermediate estimate. 
The dominant factor in computational complexity is the 
nearest neighbor search (Eq. 4). A higher value of A in the 
minimization implies two axis aligned orthogonal principle 
search directions (two spatial coordinates) in the search 
space. Therefore exact nearest neighbor search for a given 
patch can be performed in 0(N 2 ) time using adaptive k-d 
tree search structure where N is the number of pixels in the 
image. This yields sub-quadratic complexity O(Nz) for 
the overall algorithm, which is a significant improvement 
over state-of-the-art nonparametric algorithms [7], which 
involve a complex training phase and whose best-case 
complexity is quadratic ( 0(N 2 )). 

4. Results 

We test our algorithm on a number of standard im¬ 
ages, and compare it with state-of-the-art parametric as 
well as example-based image upsampling algorithms [7, 5]. 
Figure 6 indicates the significant improvements achieved 
by our method over the nonparametric scale-space search 
method advocated by Glasner and Irani [7]; our method ac¬ 
curately reproduces sharp edges, reduces haloing artifacts, 
and synthesizes photorealistic textures. Figure 8 compares 








Bicubic interpolation Scale-space search [7] Proposed layer-based method 

Figure 6. Example 4x upsampling result of a 128 x 128 image using layer-based upsampling. Our proposed method retains sharpness of 
salient edges, eliminates halos, as well as reproduces fine textural details at greatly reduced computational cost. Quality of images in this 
paper best evident when viewed full-screen on a computer monitor. 



Bicubic interpolation Scale-space search [7] Proposed layer-based method 

Figure 7. Example 4x (cropped) upsampling results of a 244 x 200 image using layer-based upsampling. 


Method 

PSNR (dB) 

SSIM 

Example-based SR [ 6 ] 

21.43 

0.6438 

Filter-based upsampling [5] 

21.30 

0.6483 

Scale-space search [7] 

21.45 

0.6829 

Proposed layer-based method 

23.93 

0.7624 


Table 1. Numerical comparison of upsampling algorithms for the 
images in Figs 6 and 8. 


the performance of our algorithm with other state-of-the- 
art algorithms, such as example-based super-resolution by 
Freeman et al. [f ] and edge-aware filter-based upsampling 
by Freedman and Fattal [1 ]. The perceptual improvement 
offered by our method is evident, particularly in detail (tex¬ 
ture) regions. Figure 10 illustrates the improved perfor¬ 
mance of our proposed algorithm over the method of Im¬ 
posed Edge Statistics [4], simultaneously obviating the need 
for a complex parameter training phase, as well as much im¬ 
proved speed of recovery. 

Figures 7 and 9 provide additional results and compar¬ 
isons with the state-of-the-art. Our method is able to repro¬ 


duce fine-scale detail (like the girl’s hair, the gentleman’s 
hat, and the koala’s fur in Figures 9 and 10). Our method 
also reduce ringing artifacts around sharp edges (such as the 
thin white ridge in Fig. 7). Please refer to the supplemental 
material for several additional comparisons and visualiza¬ 
tions. 

It is not typical in the literature to report upsampling re¬ 
sults in terms of objective numerical comparisons; the rea¬ 
sons for this is that it is often hard to define an objective 
metric of image quality, and ground truths for the stan¬ 
dard test images are not always available. Nevertheless, we 
present some numerical comparisons in Table 1 in terms 
of two popular metrics: peak SNR (in dB), and structural 
similarity index (SSIM) [1 ]. It is clear from Table 1 that 
our proposed method performs extremely favorably under 
both metrics. In particular, SSIM is widely considered to 
be a standard image quality measure for human perception; 
a higher value of SSIM indicates better performance; our 
layer-based method yields the highest SSIM value among 
all state-of-the-art upsampling algorithms. 



























Example-based SR [6] Filter-based upsampling [5] Proposed layer-based method 

Figure 8 . Example 4x upsampling results of a 128 x 128 image using layer-based upsampling: additional comparisons. 



Bicubic interpolation 


Scale-space search [ 


Filter-based Upsampling [ ] 


Figure 9. Example 3x upsampling results. 


Proposed layer-based method 


We implemented our method in MATLAB 2 and executed 
it on a Windows desktop machine with a 3.4GHz dual-core 
Intel-i7 processor. On average, our method takes less than 1 


2 The nearest-neighbor step in Eq. 4 was implemented using the open- 
source MATLAB package OpenTSTool, which consists of a library mexed 
C++ functions. 


minute to upsample an image of size 200 x 200 by a magni¬ 
fication factor of 4 in either dimension. Compared to other 
example-based upsampling and texture enhancement meth¬ 
ods, this represents an order-of-magnitude increase. A key 
reason for this improvement is that we favor local searches 
in our edge-enhancement step (Eq. 4). This greatly reduces 


































Imposed edge statistics [4] Proposed layer-based method 
Figure 10. Example 4x (cropped) upsampling results of a 266 x 
200 image using layer-based upsampling. 


computational demands, as well as makes our method easily 
parallelizable. Hence our proposed method can potentially 
be viable when implemented on a GPU. 

4.1. Comparisons to Prior Art 

Our method extends and refines aspects of various ex¬ 
isting super-resolution and detail enhancement methods. 
Primarily, we have advocated to explicitly deal with up- 
sampling of texture details, which has been the bane for 
many super-resolution algorithms. Our proposed algorithm 
bears resemblance to the scale-space search approach [ 2 , 7 ]. 
However, we recognize that the non-parametric search is 
both computationally demanding, as well as fails to repro¬ 
duce fine textures, and thus discuss methods to alleviate 
these concerns. Our algorithm also can be linked to the non¬ 
stationary filter-based upsampling method [5], which also 
uses a non-parametric search step to upsample the sharp 
edges. However, the upsampling filters used in that method 
are often hard to compute for arbitrary scaling factors. Fi¬ 
nally, we contrast our proposed method with the texture hal¬ 
lucination approach [8]. We echo their concerns that tex¬ 
tures are problematic in natural image upsampling; how¬ 
ever, we propose a universal scheme for reproducing high- 
resolution textures that does not require manual interven¬ 
tion, obviates the need for a training database, and enjoys a 
great reduction in computational complexity. 

5. Conclusions 

In contrast to previous methods for image upsampling, 
our method explicitly recognizes the presence of various 
layers in natural images; edges and fine-scale details are 
fundamentally different, and thus need to be upsampled in 
a fashion unique to their structure. There are numerous av¬ 
enues for improvement of our algorithm. The exact choice 


of the texture enhancement step £ will likely influence the 
quality of synthesized details, as well as the speed of the 
overall algorithm. We used a simple, global detail enhance¬ 
ment procedure; a localized, region-specific enhancement 
might lead to better results. We defer this to future research. 
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